On this page

Saving to Generic S3 Storage

Introduction

This tutorial is crafted to guide you through the process of saving files to any S3-compatible storage service, emphasizing flexibility and compatibility with a broad range of cloud storage options.

By completing this tutorial, you'll learn how to set up a destination object for saving files to a generic S3 location, including configuring the URL structure and necessary credentials.

Supported Protocols

Protocol	Description
s3://	S3 Protocol

1
Selecting an S3-Compatible Service

Choose any S3-compatible storage service according to your preferences and requirements (AWS S3, Qencode S3, Google Cloud Storage, DigitalOcean Spaces, Wasabi, Cloudflare R2, etc.).

2
Creating an S3 Bucket

Sign in to your chosen S3-compatible service.
Create a new storage bucket and assign it a unique name.

3
Saving Output to an S3 Bucket

In order to save your outputs to your S3 Storage, define your destination object using the following structure.

Destination Object Structure

"destination": {
  "url": "s3://endpoint/bucket/path",
  "key": "access_key",
  "secret": "secret_key",
  "permissions": "permissions"
}

Attributes of S3 Destination Object

Attribute	Description
url	Your S3 Destination URL. Includes the endpoint, port if applicable, bucket and path.
key	Your access key for the S3-compatible service.
secret	Your secret key for the S3-compatible service.
permissions	Specifies object access permissions. The default value is ‘private’ meaning the object is not publicly accessible with its URL. In case you want your objects to be accessed publicly, specify ‘public-read’ as a value for permissions.

URL Structure for S3

The url used for your Generic S3 Storage destination object consists of endpoint, port, bucket and path. Check the examples for single file and ABR formats like HLS and DASH below:

s3://[endpoint][:port]/[bucket]/[path]

When saving ABR outputs like HLS or DASH, please specify the path to a folder where the contents of the HLS or DASH stream should be located:

s3://[endpoint][:port]/[bucket]/folder

Components of URL

endpoint	The endpoint URL of your S3-compatible service.
port	Optional. The port number, if different from the default S3 port.
bucket	Your specific bucket name.
path	Path within your bucket where the file will be saved.

Destination Object Example

"destination": {
  "url": "s3://us-west.s3.qencode.com/my-bucket/folder/output.mp4",
  "key": "myAccessKeyId",
  "secret": "mySecretAccessKey",
  "permissions": "public-read",
  "storage_class": "REDUCED_REDUNDANCY"
}

"destination": {
  "url": "s3://us-west.s3.qencode.com/my-bucket/folder",
  "key": "myAccessKeyId",
  "secret": "mySecretAccessKey",
  "permissions": "public-read",
  "storage_class": "REDUCED_REDUNDANCY"
}

Working with non-HTTPS endpoints

If your S3 service does not support HTTPS and instead uses plain HTTP, set the use_tls attribute to 0:

"destination": {
  "url": "s3://example.endpoint.com/my-bucket/folder/output.mp4",
  "key": "myAccessKey",
  "secret": "mySecretKey",
  "use_tls": 0
}

Considerations for S3 Storage

Service Selection: Different S3-compatible services may offer varying features and pricing models.
Security and Access Control: Manage your keys and access permissions appropriately.
Connectivity Issues: Ensure the endpoint, bucket name, and path are correctly specified in the URL.
Authentication Errors: Double-check your access and secret keys for accuracy.
Explore advanced functionalities and integration possibilities of your chosen S3-compatible service.